Skip to content

feat(validation): oneOf near-miss validator hints + issues[].hint on every VALIDATION_ERROR#476

Merged
bokelley merged 1 commit intomainfrom
bokelley/feat-oneof-hints
May 3, 2026
Merged

feat(validation): oneOf near-miss validator hints + issues[].hint on every VALIDATION_ERROR#476
bokelley merged 1 commit intomainfrom
bokelley/feat-oneof-hints

Conversation

@bokelley
Copy link
Copy Markdown
Contributor

@bokelley bokelley commented May 3, 2026

Summary

When a discriminated-union (oneOf) shape fails validation because the caller used the wrong key as the discriminator — the v3 reference-seller pricing_options regression where an LLM client submits {"type": "cpm", ...} instead of {"pricing_model": "cpm", ...} — emit an additive hint field on the VALIDATION_ERROR issue:

Looks like you may have meant the 'cpm' variant. Use 'pricing_model' instead of 'type' as the discriminator.

  • New adcp.validation.oneof_hints.compute_oneof_hint heuristic — pure, schema-driven, no Pydantic dependency (the framework's validator is jsonschema-based).
  • ValidationIssue gains an optional hint: str | None = None field.
  • _format_error now accepts the validator's resolved schema + payload, computes the hint when keyword == oneOf, and attaches it.
  • _issue_to_wire only writes hint to the wire dict when populated, so clients that ignore the new field see exactly the pre-hint envelope shape — additive change.

The hint is best-effort: when no clear winner exists across variants, no hint is emitted (silent is better than misleading).

Heuristic

  1. Walk to the schema's oneOf keyword via the failing issue's absolute_schema_path.
  2. Detect the discriminator field — a property pinned by const in at least two variants. No discriminator → no hint.
  3. Score each variant by (const_match, required_present, total_present). The strongest signal is whether any value in the payload matches a variant's discriminator const (the wrong-key case); shape match is the tiebreaker. Tie at the top → no hint.
  4. Identify the wrong key in the payload — first key whose value matches the best variant's const, or first key not declared by any variant's properties.

Projection point

adcp.validation.schema_validator._format_error projects every jsonschema ValidationError into a ValidationIssue. Two callers (validate_request / validate_response) now forward validator.schema and the payload through. The wire envelope is produced by _issue_to_wire (used by both SchemaValidationError.details and build_adcp_validation_error_payload in schema_errors.py).

Divergences from JS commit e7f8e228

@adcp/sdk@6.7's e7f8e228 lands three things:

  • #1338 SHAPE-GOTCHAS docs (skill catalog only — not portable to Python SDK).
  • #1325 TS codegen tightening (TS-specific).
  • #1337 validator change to exclude not-failure variants when picking the best surviving oneOf variant — about Success/Error response envelope mutual exclusion.

The hint-emitting near-miss feature described by issue #460 is not implemented as a single named function in JS; it's the design intent the issue spec calls out. This PR builds it for Python directly, using the regression class (type vs pricing_model on pricing_options) as the load-bearing test case. The JS not-failure exclusion is orthogonal and can be ported separately if needed.

Test plan

  • pytest tests/test_oneof_hints.py -v — 12 new tests, all pass
    • The v3 ref-seller regression (type instead of pricing_model) produces a hint naming the cpm variant + pricing_model vs type
    • Tested both inline-schema and end-to-end against the bundled get_products schema
    • No clear winner → no hint
    • No discriminator in schema → no hint
    • Discriminator key present but invalid value → no hint (standard message is more accurate)
    • Variant with multiple options → picks the one whose const matches caller's value
  • pytest tests/ — 3442 passed, 27 skipped, 1 xfailed; no regressions
  • ruff check clean on touched files
  • mypy clean on touched implementation files
  • Pre-commit hooks (black, ruff, mypy, bandit) all pass

Closes #460. Refs #452.

🤖 Generated with Claude Code

@bokelley bokelley force-pushed the bokelley/feat-oneof-hints branch from 280de10 to dcc5ab4 Compare May 3, 2026 14:31
…every VALIDATION_ERROR

When a discriminated-union (`oneOf`) shape fails validation because the
caller used the wrong key as the discriminator (the v3 reference-seller
`pricing_options` regression: `{"type": "cpm", ...}` instead of
`{"pricing_model": "cpm", ...}`), an additive `hint` field on the
`VALIDATION_ERROR` issue names the closest matching variant and the
wrong / expected discriminator keys:

    Looks like you may have meant the 'cpm' variant. Use
    'pricing_model' instead of 'type' as the discriminator.

The hint is best-effort: when no clear winner exists across variants,
no hint is emitted (silent is better than misleading). Clients that
ignore the new field behave exactly as before — `hint` is only added
to the wire envelope's `issues[i]` dict when populated.

Heuristic (in `adcp.validation.oneof_hints.compute_oneof_hint`):

1. Walk to the schema's `oneOf` keyword via the failing issue's
   `absolute_schema_path`.
2. Detect the discriminator field — a property pinned by `const` in
   at least two variants. No discriminator -> no hint.
3. Score each variant by `(const_match, required_present, total_present)`:
   the strongest signal is whether any value in the payload matches a
   variant's discriminator `const` (the wrong-key case); shape match
   is the tiebreaker. Tie at the top -> no hint.
4. Identify the wrong key in the payload — first key whose value
   matches the best variant's `const`, or first key not declared by
   any variant's `properties`.

Plumbed through `validate_request` / `validate_response` (which now
forward the validator's resolved schema + payload to `_format_error`)
and surfaced via the existing `_issue_to_wire` helper used by both
`SchemaValidationError.details` and
`build_adcp_validation_error_payload`.

Closes #460. Refs #452.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@bokelley bokelley force-pushed the bokelley/feat-oneof-hints branch from dcc5ab4 to 38b958f Compare May 3, 2026 14:32
@bokelley bokelley merged commit 9985086 into main May 3, 2026
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(validation): oneOf near-miss validator hints + issues[].hint on every VALIDATION_ERROR

1 participant